Reporting transparency and completeness in trials: Paper 4 - reporting of randomised controlled trials conducted using routinely collected electronic records – room for improvement

Objective To describe characteristics of randomized controlled trials (RCTs) conducted using electronic health records (EHRs), including completeness and transparency of reporting assessed against the 2021 CONSORT Extension for RCTs Conducted Using Cohorts and Routinely Collected Data (CONSORT-ROUTINE) criteria. Study Design MEDLINE and Cochrane Methodology Register were searched for a sample of RCTs published from 2011–2018. Completeness of reporting was assessed in a random sample using a pre-defined coding form. Results Of the 183 RCT publications identified, 122 (67%) used EHRs to identify eligible participants, 139 (76%) used the EHR as part of the intervention and 137 (75%) to ascertain outcomes. When 60 publications were evaluated against the CONSORT 2010 item and the corresponding extension for the 8 modified items, four items were 'adequately reported' for most trials. Five new reporting items were identified for the CONSORT-ROUTINE extension; when evaluated, one was ‘adequately reported’, three were reported ‘inadequately or not at all’, the other ‘partially’. There were, however, some encouraging signs with adequate and partial reporting of many important items, including descriptions of trial design, the consent process, outcome ascertainment and interpretation. Conclusion Aspects of RCTs using EHRs are sub-optimally reported. Uptake of the CONSORT-ROUTINE Extension may improve reporting.


Appendix 3. Coding manual for the completeness and transparency of reporting
Did the authors clearly describe a (1) structured summary of (2) trial design, (3) methods, (4)

results, and (5) conclusions
Did the authors only report one, two, three or four element(s) of this item and not all five elements of the item?
Did the authors not describe a structured summary of trial design, methods, results and conclusions?
1b Structured summary of trial design, methods, results, and conclusions (for specific guidance see CONSORT for abstracts). Specify that a cohort or routinely collected data were used to conduct the trial and, if applicable, provide the name of the cohort or routinely collected database(s) (Modified) Did the authors specify that a cohort or routinely collected data were used to conduct the trial?
Did the authors describe methods that would typically require routinely collected data for components of the trials but not specify they used routinely collected data?
Did the authors not specify that routinely collected data were used to conduct the trial?

Introduction
Trial design 3a Description of trial design (such as parallel, factorial) including allocation ratio Did the authors clearly describe the trial design including allocation ratio?
All other cases, where applicable.
Did the authors not describe the trial design including allocation ratio?
Description of trial design (such as parallel, factorial) including allocation ratio, that a cohort or routinely collected database(s) was used to conduct the trial (such as electronic health record, registry) and how the data were used within the trial (such as identification of eligible trial participants, trial outcomes) (Modified) Did the authors clearly mention the (1) routinely collected database(s) that were used within the trial and (2) how the data were used within the trial (i.e. identification of participants, outcome measurement, other)?
Did the authors only report one element of this item and not both elements of the item?
Did the authors not describe the routinely collected database(s) that were used within the trial and not describe how the data were used within the trial (i.e. identification of participants, outcome measurement, other)?

ROUTINE-
Name, if applicable, and description of Did the authors clearly (1) Did the authors only Did the authors not name 1 the cohort or routinely collected database(s) used to conduct the trial, including information on the setting (such as primary care), locations, and dates, (such as periods of recruitment, follow-up, and data collection) (New) name and (2) describe the routinely collected database(s) and (3) provide information on the setting, locations, and relevant dates (e.g. periods of recruitment, follow-up, and data collection)?
report one or two element(s) of this item and not all three elements of the item? and describe the routinely collected database(s) and not provide information on the setting, locations, and relevant dates (e.g. periods of recruitment, follow-up, and data collection)?

ROUTINE-2
Eligibility criteria for participants in the cohort or routinely collected database(s) (New) Did the authors clearly describe the eligibility criteria for the routinely collected database(s)?
All other cases, where applicable.
Did the authors not describe all eligibility criteria for the routinely collected database(s)?

ROUTINE-3
State whether the study included person-level, institutional-level, or other data linkage across two or more databases and, if so, linkage techniques and methods used to evaluate completeness and accuracy of linkage (New) Did the authors clearly state whether the study included (1) person-level, institutional-level, or other data linkage across two or more databases and (2)  Eligibility criteria for trial participants, including information on how to access the list of codes and algorithms used to identify eligible participants, information on accuracy and completeness of data used to ascertain eligibility, and methods used to validate accuracy and completeness (e.g., monitoring, adjudication), if applicable (Modified) Did the authors provide information on (1) how to access the lists of codes and algorithms used to identify participants, including (2) methods used to assess accuracy and completeness, if applicable?
Did the authors only report one element of this item and not both elements of the item?
Did the authors not provide information on how to access the lists of codes and algorithms used to identify participants, and not provide the methods used to assess accuracy and completeness?
The trial did not use routinely collected data to identify participants

ROUTINE-4
Describe whether and how consent was obtained (New) Did the authors describe clearly whether and how consent was obtained?
All other cases, where applicable.
Did the authors not describe whether and how consent was obtained?
Outcomes 6a Completely defined prespecified primary and secondary outcome Did the authors clearly define the pre-specified primary and secondary Did the authors only define the prespecified primary and Did the authors not define the pre-specified primary and secondary outcome measures measures, including how and when they were assessed outcome measures, including how and when they were assessed? secondary outcome measures but not how and when they were assessed, or did they describe how and when outcomes were assessed but not the measures?
and not define how and when they were assessed?
Completely defined pre-specified primary and secondary outcome measures, including how and when they were ascertained and the cohort or routinely collected database(s) used to ascertain each outcome (Modified) Did the authors clearly describe the routinely collected database(s) used to ascertain each outcome?
All other cases, where applicable.
Did the authors not describe the routinely collected database(s) used to ascertain each outcome?
The trial did not use routinely collected data to ascertain the outcome

ROUTINE-5
Information on how to access the list of codes and algorithms used to define or derive the outcomes from the cohort or routinely collected database(s) used to conduct the trial, information on accuracy and completeness of outcome variables, and methods used to validate accuracy and completeness (e.g., monitoring, adjudication), if applicable (New) Did the authors clearly (1) describe information on how to access the list of codes and algorithms used to define or derive the outcomes from the routinely collected database(s), (2) including methods used to assess accuracy and completeness?
Did the authors only report one element of this item and not both elements of the item?
Did the authors not describe information on how to access the list of codes and algorithms used to define or derive the outcomes from the routinely collected database(s), and not describe the methods used to assess accuracy and completeness?
The trial did not use routinely collected data to ascertain the outcome Allocation concealment mechanism 9 Mechanism used to implement the random allocation sequence (such as sequentially numbered containers), describing any steps taken to conceal the sequence until interventions were assigned Mechanism used to implement the random allocation sequence (such as embedding an automated randomiser within the cohort or routinely collected database(s)), describing any steps taken to conceal the sequence until interventions were assigned (Modified) Did the authors clearly describe the mechanism used to implement the random allocation sequence (such as embedding an automated randomiser within the cohort or routinely collected database(s)), describing any steps taken to conceal the sequence until interventions were assigned?
All other cases, where applicable Did the authors not describe the mechanism used to implement the random allocation sequence (such as embedding an automated randomiser within the cohort or routinely collected database(s)), describing any steps taken to conceal the sequence until interventions were assigned?
Participant flow (a diagram is  strongly  recommended) 13a For each group, the numbers of participants who were randomly assigned, received intended treatment, and were analysed for the primary outcome Did the authors define clearly for each group, (1) the number of participants who were randomly assigned, (2) received intended treatment and (3) were analysed for the primary outcome?
Did the authors only report one or two elements of this item and not all three elements of the item or only presented this information for one group?
Did the authors not describe clearly for each group, the number of participants who were randomly assigned, and not received intended treatment and not were analysed for the primary outcome?
For each group, the number of participants in the cohort or routinely collected database(s) used to conduct the trial and the numbers screened for eligibility, randomly assigned, offered and accepted interventions (e.g., cohort multiple RCTs), received intended treatment, and analysed for the primary outcome (Modified) Did the authors clearly define, for each group, the number of participants in the routinely collected database(s) used to conduct the trial and the numbers screened for eligibility, randomly assigned, received intended treatment, and analysed for the primary outcome?
Did the authors only report some, but not all, elements of this item?
Did the authors not define, for each group, the number of participants in the routinely collected database(s) used to conduct the trial and not define the numbers screened for eligibility, randomly assigned, received intended treatment, and analysed for the primary outcome Interpretation consistent with results, balancing benefits and harms, and considering other relevant evidence, including the implications of using data that were not collected to answer the trial research questions (Modified) Did the authors describe the implications of using data that were not collected to answer the trial research questions?

Discussion
Note: the authors have to report information about the issues of using routine data.
All other cases, where applicable Did the authors not describe the implications of using data that were not collected to answer the trial research questions? Sources of funding and other support for both the trial and the cohort or routinely collected database(s), role of funders (Modified)

Other information
Did the authors clearly describe the sources of funding for the database(s) and trial and the role of the funder of the trial?
Did the authors only report some, but not all, elements of this item?
Did the authors not describe the sources of funding for routinely collected database(s) and trial and not describe the role of the funder of the trial?